Recap of Part 1
In Part 1 of this workshop, we attempted to predict the value of a
single, continuous outcome variable hourly_wage based on a
range of input features, including age, location, and industry.
Specifically, we covered:
Train-test splits: partitioning the data set into a training set
used to train our model, and a test set used to evaluate the model’s
performance.
The bias-variance trade-off: the balance between the model’s
ability to capture the true underlying patterns of the training data
(bias) and its flexibility to learn from specific instances within that
data (variance). This trade-off matters because it directly affects the
generalization ability of the model to new, unseen data.
Pre-processing: cleaning and transforming raw data into a
suitable format for analysis, ensuring the data is free of errors,
inconsistencies, and irrelevant information. It often includes coding
categorical variables and normalizing numerical values to reduce bias
and improve model accuracy.
Today, we will apply and expound upon these principles to develop
models to predict categorical variables. This process is called
‘classification.’
Other Tasks in Supervised Machine Learning: Classification
Thus far, we’ve spent most of our time working through regression
problems (i.e., predicting a continuous outcome variable). Let’s switch
to a new task: classification. In classification, we aim to predict one
of a group of values. For example, predicting a qualitative response is
considered classification because we assign the observation to a
category (or class).
Like regression, classification is also a supervised learning
technique because we have a set of labeled training data that we can use
to build a classifier. Now, however, we have access to different
techniques because the structure of our outcome variable is
categorical.
📝 Poll 1: Classification vs. Regression
Identify which of the following are classification problems in
machine learning.
An advertiser is interested in the relationship between age and
the number of hours of YouTube consumed.
A medical testing company conducts a procedure to determine
whether a person has a cancer diagnosis.
A researcher is interested in the effect of an education
intervention on students’ test scores.
A software engineer is designing an algorithm to detect whether
an email is spam or not.
A political scientist wants to classify Twitter posts as positive
or negative.
Solution 1: (2), (4), and (5).
Train-Test Split
Recall from Part 1 that a critical first step in machine learning is
partitioning our data into training and test sets.
🔔 Question 1: Train-Test Split
A researcher used the entire vote2020 data set to train
a model that resulted in impressively high accuracy during evaluation.
However, upon deploying this model to predict voter turnout for an
upcoming election, the predictions significantly diverged from actual
turnout, with many discrepancies in who was predicted to vote versus who
actually voted. The model’s near-perfect performance in development
starkly contrasted its poor real-world prediction outcomes, indicating a
potential oversight in the model training and evaluation process. Why
might that be?
Answer 1: The key issue is that the model was never
tested on unseen data. By using the entire data set for training, the
model essentially “memorized” the data, including any noise or patterns
specific to that set of individuals. This led to overfitting, where the
model excelled at predicting the training data but failed to generalize
to new, unseen data. Since the model’s performance was only evaluated on
data it had already seen, its apparent accuracy was misleading, giving a
false sense of confidence in its predictive capabilities.
Now that we have a better sense of the data, let’s go ahead and split
vote2020 into training and test sets:
# Perform splits
vote2020$voted <- as.factor(vote2020$voted)
vote_split <- initial_split(vote2020, prop = 0.75)
vote_train <- training(vote_split)
vote_test <- testing(vote_split)
Pre-Processing
In our example from Part 1, we prepared our data for analysis by
recoding categorical variables and normalizing numeric ones using
step_dummy(all_nominal_predictors()) and
step_normalize(), respectively.
In that example, we also dropped rows that had any missing values
across variables. Let’s try another example, in which we don’t
omit samples that have missing values, but instead perform
imputation, in which we replace those missing values according
to certain criteria. There are various kinds of imputation,
including:
For example, whenever we have a missing value for the
occupation, we can replace that missing value with the most
common occupation. This is called mode imputation.
Or, we could replace a missing numerical predictor (e.g., age)
using the median across all the samples. This is called median
imputation.
There are other ways to impute, but these are good starting points.
The way to perform imputation in a recipe is via the
step_impute_ functions.
🥊 Challenge 2: Creating a Recipe
Using the same logic from Part 1, create your own recipe called
voterecipe that lays out the steps for pre-processing the
data. Instead of dropping rows with missing values, use step_impute_median
to impute numeric variables and step_impute_mode
to impute categorical variables.
voterecipe <-
recipe(voted ~ ., data = vote_train) %>%
step_impute_median(all_numeric_predictors()) %>%
step_impute_mode(all_nominal_predictors()) %>%
step_dummy(all_nominal_predictors())
This pre-processing will allow us to take advantage of samples with
missing data, even if it comes at a little cost to accuracy. Imputation
is often a necessary step, since it’s common to have missing data.
🔔 Question 2: Imputation
Notice that we have only applied our recipe to the training data so
far. Explain why it is important to perform imputation separately on the
training and test sets rather than imputing missing values before
splitting the data set. Consider why we don’t want the data we train our
model on to be exposed to our test data set in the first place.
Answer 2: We want to keep our training and test sets
separate. If we impute missing values in the entire data set using the
mean, the mean calculation will include information from both the
training and test sets. This means the model gets indirectly exposed to
information from the test set during training, which can lead to overly
optimistic performance estimates and a model that may not perform as
well on truly unseen data. This concept is referred to as ‘data
leakage.’
Developing and Evaluating Models
Now that we’ve performed exploratory analyses, a train-test split,
and pre-processing, let’s go ahead and train our model. We will create a
classifier (i.e., a model that predicts membership in a group) with
logistic regression.
Algorithm 1: Logistic Regression
Machine learning practitioners often recommend logistic regression as
a starting model when predicting a binary outcome or probability. For
example, if we are estimating the relationship between mortality and
income, we can estimate the probability of mortality given a change in
income.
We write this as \(P[M|I]\) and the
values will range between 0 and 1. We can make a prediction for any
given value of income on the mortality outcome. Normally, we establish a
threshold for prediction. For example, we might predict death (M) where
\(P[M|i] > 0.5\).
Logistic regression is a generalized linear model where we model the
probability function using the logistic function, an s-shaped curve
sometimes called a sigmoid function. The general function is: \(p(Y= 1|X)\). In the example below, the
green data points will be classified as 0 because \(p\) < 0.5; the orange points will be
classified as 1 because \(p\) >
0.5.
Logistic Regression
Here’s how to fit classification problems in tidymodels
using logistic regression.
In tidymodels, creating a logistic regression follows
the exact same procedure as a linear regression. This time, however, we
will use logistic_reg() to initiate the function. Let’s
create the model:
# Create model
logistic_model <- logistic_reg(mode = "classification")
Now, let’s fit the model on the training data. We first create a
workflow that lists instructions for pre-processing and the model we
want to use, and then apply it to the training data.
vote_wflow <- workflow() %>%
add_recipe(voterecipe) %>%
add_model(logistic_model)
vote_fit <- fit(vote_wflow, vote_train)
vote_fit %>% tidy()
Finally, let’s use augment from the yardstick package to
obtain the predictions, and take a look at them:
logistic_predictions <- augment(vote_fit, new_data = vote_test)
logistic_predictions[,14:17] # look at last 4 columns
🔔 Question 3: Analyzing Output
Notice that four new columns have appeared in our data set. What do
these values mean? What are they telling us?
Answer 3: .pred_class corresponds to
the value our algorithm is predicting, i.e., whether the person voted in
the election or not. This was determined using the two variables
.pred_0 and .pred_1, which represent the
predictions from the logistic regression. If membership in group 1
(i.e., voted = 1) is greater than .5, then .pred_class will
take a value of 1.
To evaluate the model, we’ll use the accuracy
function:
accuracy(logistic_predictions, truth = voted, estimate = .pred_class, type="class")
We predicted the likelihood that someone voted in the last election
with an accuracy of about 81% - not bad!
We can also create what’s called a ‘confusion matrix’ to see how well
our model did at predicting each class. A confusion matrix is helpful if
you care about false positives (predicting 1 when the true value is 0)
and false negatives (predicting 0 when the true value is 1). In the
context of testing for Covid-19, we might want care more about failing
to diagnose a person with the virus (i.e., a false negative) because the
stakes are higher.
logistic_predictions %>%
conf_mat(truth = voted, estimate = .pred_class)
Truth
Prediction 0 1
0 802 514
1 2753 13252
So far we have trained a simple logistic regression model, there are
many other options in our machine learning arsenal to get a better
prediction.
Hyperparameter Tuning and Regularization
Notice that until now, we have used the default settings for the
linear and logistic regressions we have run. Looking at the
(documentation)[https://www.rdocumentation.org/packages/parsnip/versions/0.0.0.9001/topics/logistic_reg]
for logistic regression, we see that there are many arguments that we
left blank when we initialized our model above. These arguments, also
known as ‘parameters’, correspond to statistical choices about how the
model should operate or be structured.
Some of these hyperparameters include ‘engine’ and ‘penalty’.
Engine: Different engines can implement regression through
various algorithms or computational approaches. When you specify an
engine for logistic regression in R, you’re choosing the particular set
of algorithms and optimizations that will be used to train your
model.
Penalty: “penalty” refers to a regularization technique used to
prevent overfitting by discouraging overly complex models. It does this
by adding a penalty to the loss function for large coefficients. Common
penalties are L1 (Lasso), which can shrink some coefficients to zero
(thus performing feature selection), and L2 (Ridge), which shrinks all
coefficients toward zero but typically doesn’t set any to exactly zero.
The penalty helps in creating simpler, more generalizable models that
perform better on unseen data by prioritizing the most influential
features and reducing the model’s sensitivity to the training data’s
noise.
For example, we can add a ‘penalty’ on the size of the coefficients
of a model and reduce the likelihood of overfitting. Lasso, ridge, and
elastic net are different types of penalties that greatly reduce or
shrink to zero coefficients on variables that are picking up on a lot of
noise. The broad technique of reducing overfitting is called
‘regularization.’
🥊 Challenge 3: Changing Hyperparameters
We have reproduced the original code from above that trained a basic
classifier on our voting data set without changing any of the
hyperparameters, as well as the code that obtains predictions. Re-run
this code several times, but each time change the hyperparameters in the
model specification. For penalty, include a non-negative number; and for
engine, select either “glmnet” or “glm”. How does the accuracy
change?
chal3_model <- logistic_reg(mode = "classification",
engine="glmnet",
penalty=0)
chal3_wflow <- workflow() %>%
add_recipe(voterecipe) %>%
add_model(chal3_model)
chal3_fit <- fit(chal3_wflow, vote_train)
chal3_predictions <- augment(chal3_fit, new_data = vote_test)
accuracy(chal3_predictions, truth = voted, estimate = .pred_class, type="class")
It’s nice that we can do different types of regularization, but how
do we know what value of the penalty coefficient to pick? In machine
learning, this value - which we choose before fitting the model - is
known as a hyperparameter. Since hyperparameters are chosen
before we fit the model, we can’t just choose them based off
the training data. So, how should we go about conducting
hyperparameter tuning: identifying the best
hyperparameter(s) to use?
Let’s think back to our original goal. We want a model that
generalizes to unseen data. So, ideally, the choice of the
hyperparameter should be such that the performance on unseen data is the
best. We can’t use the test set for this, but what if we had another set
of held-out data?
Hyperparameter tuning
Cue hyperparameter tuning! Hyperparameter tuning is crucial in
machine learning because it directly impacts the performance and
effectiveness of models. By fine-tuning hyperparameters, practitioners
can optimize models to achieve higher accuracy, better generalize to
unseen data, and prevent issues like overfitting or underfitting. This
process allows for the customization of models to specific data sets and
objectives, enabling the discovery of the best configuration for a given
problem.
Choosing Hyperparameters: Validation Sets
This is the basis for a validation set. If we had
extra held-out data set, we could try a bunch of hyperparameters on the
training set, and see which one results in a model that performs the
best on the validation set. We then would choose that hyperparameter,
and use it to refit the model on both the training data and validation
data. We could then, finally, evaluate on the test set.
Validation set
Cross Validation
We just formulated the process of choosing a hyperparameter with a
single validation set. However, there are many ways to perform
validation. The most common way is cross-validation.
Cross-validation is motivated by the concern that we may not choose the
best hyperparameter if we’re only validating on a small fraction of the
data. If the validation sample, just by chance, contains specific data
samples, we may bias our model in favor of those samples, and limit its
generalizability.
So, during cross-validation, we effectively validate on the
entire training set, by breaking it up into folds. Here’s the
process: We can use a process called cross-validation to do select the
best
- Perform a train-test split, as you normally would.
- Choose a number of folds - the most common is \(K=5\) - and split up your training data
into those equally sized “folds”.
- For each hyperparameter you want to tune, specify the possible
values it could take. Then, for each hyperparameter:
- For each value of that hyperparameter, we’re going
to fit \(K\) models. Let’s assume \(K=5\). The first model will be fit on Folds
2-5, and validated on Fold 1. The second model will be fit on Folds 1,
3-5, and validated on Fold 2. This process continues for all 5
splits.
- The performance of each value of that hyperparameter is summarized
by the average predictive performance on all 5 held-out folds. We then
choose the hyperparameter value that had the best average
performance.
- We can then refit a new model to the entire training set, using our
chosen hyperparameter(s). That’s our final model - evaluate it on the
test set!
cross-validation
Hyperparameter Tuning and Cross Validation in Practice
We need to do two things:
- Decide to perform hyperparameter tuning on the penalty value,
and
- Do so using cross-validation.
The tidymodels suite has two packages to help us with
these steps: tune and rsample.
Let’s illustrate both these packages in the classification example.
We already have a recipe set up:
voterecipe
── Recipe ─────────────────────────────────────────────────────────────────────────────
── Inputs
Number of variables by role
outcome: 1
predictor: 13
── Operations
• Median imputation for: all_numeric_predictors()
• Mode imputation for: all_nominal_predictors()
• Dummy variables from: all_nominal_predictors()
When specifying the model, however, we’re going to do something
slightly different:
tuned_logistic_model <- logistic_reg(
penalty = tune(),
engine = "glmnet")
We passed in a function called tune(). This signals to
tidymodels that we’d like to tune this hyperparameter. How
do we indicate what values we should test during tuning? We’re going to
keep things simple and focus on the most basic choice of tuning: a grid
search. In this case, we specify a range of values, and then test every
single one for the hyperparameter. We use the grid_regular
function for this procedure:
# Create grid of parameters
cv_grid <- grid_regular(
penalty(range = c(-5, 5)), # select penalty values with -5 and 5
levels=10) # pick 10 values between -5 and 5
print(cv_grid)
🔔 Question 4: Analyzing a Grid
We have a hyperparameter grid with 10 rows. What does each of these
signify?
Solution 4: Each is a potential set of
hyperparameters, or arguments, for our logistic regression. R will try
each combination to find the one that generates the most accurate
prediction. As we add more hyperparameters, the combinations will
increase exponentially.
🎬 Demo: Performing Hyperparameter Tuning with
Cross Validation
Next, we need to specify how we will perform cross-validation. From
the rsample package, we can use the function
vfold_cv to create the training folds. In this case,
v is what’s used for “K”.
vote_folds <- vfold_cv(vote_train, v = 5)
vote_folds
# 5-fold cross-validation
We have a tuned model, a grid of hyperparameters, and a set of folds.
We create our workflow as before: we input the recipe (i.e., the set of
pre-processing instructions) and the model specification. But, to train
the workflow, we use the tune_grid function. All the pieces
we’ve created are passed into this function:
# Create workflow
vote_wflow <- workflow() %>%
add_recipe(voterecipe) %>%
add_model(tuned_logistic_model)
# Tune and fit models with tune_grid()
vote_cv_fit <- tune_grid(
# The workflow
vote_wflow,
# The folds we created
resamples = vote_folds,
# The grid of hyperparameters
grid = cv_grid)
There are some nice plotting functions we can use to visualize how
the performance varies as a function of the regularization. For example,
check out the autoplot() function:
autoplot(vote_cv_fit,
metric="accuracy")

Each of the ten points on this graph corresponds to a set of
hyperparameters. We can see that the third combination yields the
highest accuracy of about 81.5%. What does this tell us about how much
regularization we should use?
Instead of picking the combination of hyperparamters that yields the
best accuracy by eye, we can automate this procedure. The
select_best function will do this for us:
# Select best metric according to accuracy
vote_cv_best <- select_best(vote_cv_fit, metric = "accuracy")
vote_cv_best
Cross-validation selected a penalty value of 0.0017 as the best
hyperparameter for our model. What do we do at this point?
Recall that, during cross-validation, we split up the data and
examine performance across many folds. Now that we know what is likely
the best penalty, we can re-train on the entire training set to
produce our final model.
We do this with the finalize_workflow function.
# Get our final model and finalize workflow
cv_final <- vote_wflow %>%
finalize_workflow(parameters = vote_cv_best) %>% # use the best parameters
fit(data = vote_train)
cv_final %>% tidy()
And lastly, we’ll examine the performance on the test set:
cv_predictions <- augment(cv_final, new_data = vote_test)
accuracy(cv_predictions, truth = voted, estimate = .pred_class, type="class")
🔔 Question 5: Accuracy on the Test Set
Notice our final accuracy on the test is about 80.85%, whereas our
accuracy on the training set was higher at 81.5%. Why is our test
accuracy lower?
Solution 5: We selected our hyperparameters (i.e.,
the penalty function) to yield the highest output specifically
on our training set.
Hyperparameter Tuning vs. Default Arguments
If we compare this to the original model we fit earlier where we made
a guess at the best value for the penalty hyperparameter, it reveals
that we that we can fit better models via cross-validation than with
just a single training set. This is reflected in the higher accuracy
compared with the model where we used the default arguments of
logistic_reg().
LS0tCnRpdGxlOiAiUiBNYWNoaW5lIExlYXJuaW5nIFBpbG90IChEYXkgMikiCm91dHB1dDogaHRtbF9ub3RlYm9vawplZGl0b3Jfb3B0aW9uczogCiAgbWFya2Rvd246IAogICAgd3JhcDogNzIKLS0tCgojIFJlY2FwIG9mIFBhcnQgMQoKSW4gUGFydCAxIG9mIHRoaXMgd29ya3Nob3AsIHdlIGF0dGVtcHRlZCB0byBwcmVkaWN0IHRoZSB2YWx1ZSBvZiBhCnNpbmdsZSwgY29udGludW91cyBvdXRjb21lIHZhcmlhYmxlIGBob3VybHlfd2FnZWAgYmFzZWQgb24gYSByYW5nZSBvZgppbnB1dCBmZWF0dXJlcywgaW5jbHVkaW5nIGFnZSwgbG9jYXRpb24sIGFuZCBpbmR1c3RyeS4gU3BlY2lmaWNhbGx5LCB3ZQpjb3ZlcmVkOgoKLSAgIFRyYWluLXRlc3Qgc3BsaXRzOiBwYXJ0aXRpb25pbmcgdGhlIGRhdGEgc2V0IGludG8gYSB0cmFpbmluZyBzZXQKICAgIHVzZWQgdG8gdHJhaW4gb3VyIG1vZGVsLCBhbmQgYSB0ZXN0IHNldCB1c2VkIHRvIGV2YWx1YXRlIHRoZSBtb2RlbCdzCiAgICBwZXJmb3JtYW5jZS4KCi0gICBUaGUgYmlhcy12YXJpYW5jZSB0cmFkZS1vZmY6IHRoZSBiYWxhbmNlIGJldHdlZW4gdGhlIG1vZGVsJ3MgYWJpbGl0eQogICAgdG8gY2FwdHVyZSB0aGUgdHJ1ZSB1bmRlcmx5aW5nIHBhdHRlcm5zIG9mIHRoZSB0cmFpbmluZyBkYXRhIChiaWFzKQogICAgYW5kIGl0cyBmbGV4aWJpbGl0eSB0byBsZWFybiBmcm9tIHNwZWNpZmljIGluc3RhbmNlcyB3aXRoaW4gdGhhdAogICAgZGF0YSAodmFyaWFuY2UpLiBUaGlzIHRyYWRlLW9mZiBtYXR0ZXJzIGJlY2F1c2UgaXQgZGlyZWN0bHkgYWZmZWN0cwogICAgdGhlIGdlbmVyYWxpemF0aW9uIGFiaWxpdHkgb2YgdGhlIG1vZGVsIHRvIG5ldywgdW5zZWVuIGRhdGEuCgotICAgUHJlLXByb2Nlc3Npbmc6IGNsZWFuaW5nIGFuZCB0cmFuc2Zvcm1pbmcgcmF3IGRhdGEgaW50byBhIHN1aXRhYmxlCiAgICBmb3JtYXQgZm9yIGFuYWx5c2lzLCBlbnN1cmluZyB0aGUgZGF0YSBpcyBmcmVlIG9mIGVycm9ycywKICAgIGluY29uc2lzdGVuY2llcywgYW5kIGlycmVsZXZhbnQgaW5mb3JtYXRpb24uIEl0IG9mdGVuIGluY2x1ZGVzCiAgICBjb2RpbmcgY2F0ZWdvcmljYWwgdmFyaWFibGVzIGFuZCBub3JtYWxpemluZyBudW1lcmljYWwgdmFsdWVzIHRvCiAgICByZWR1Y2UgYmlhcyBhbmQgaW1wcm92ZSBtb2RlbCBhY2N1cmFjeS4KClRvZGF5LCB3ZSB3aWxsIGFwcGx5IGFuZCBleHBvdW5kIHVwb24gdGhlc2UgcHJpbmNpcGxlcyB0byBkZXZlbG9wIG1vZGVscwp0byBwcmVkaWN0IGNhdGVnb3JpY2FsIHZhcmlhYmxlcy4gVGhpcyBwcm9jZXNzIGlzIGNhbGxlZAonY2xhc3NpZmljYXRpb24uJwoKIyBJbnN0YWxsIFBhY2thZ2VzIGFuZCBMb2FkIERhdGEKCkJlZm9yZSB3ZSBkaXZlIGludG8gdG9kYXkncyBtYXRlcmlhbCwgbGV0J3MgbG9hZCBvdXIgbGlicmFyaWVzCmB0aWR5bW9kZWxzYCBhbmQgYHRpZHl2ZXJzZWAgbGlicmFyaWVzLgoKYGBge3IgaW5zdGFsbCwgaW5jbHVkZT1GQUxTRX0KbGlicmFyeSh0aWR5bW9kZWxzKQpsaWJyYXJ5KHRpZHl2ZXJzZSkKCiMgUHJlZmVyIHRpZHltb2RlbHMgZnVuY3Rpb25zIGluIGFueSBjYXNlIG9mIG5hbWUgY29uZmxpY3QKdGlkeW1vZGVsczo6dGlkeW1vZGVsc19wcmVmZXIoKSAKYGBgCgojIE90aGVyIFRhc2tzIGluIFN1cGVydmlzZWQgTWFjaGluZSBMZWFybmluZzogQ2xhc3NpZmljYXRpb24KClRodXMgZmFyLCB3ZSd2ZSBzcGVudCBtb3N0IG9mIG91ciB0aW1lIHdvcmtpbmcgdGhyb3VnaCByZWdyZXNzaW9uCnByb2JsZW1zIChpLmUuLCBwcmVkaWN0aW5nIGEgY29udGludW91cyBvdXRjb21lIHZhcmlhYmxlKS4gTGV0J3Mgc3dpdGNoCnRvIGEgbmV3IHRhc2s6IGNsYXNzaWZpY2F0aW9uLiBJbiBjbGFzc2lmaWNhdGlvbiwgd2UgYWltIHRvIHByZWRpY3Qgb25lCm9mIGEgZ3JvdXAgb2YgdmFsdWVzLiBGb3IgZXhhbXBsZSwgcHJlZGljdGluZyBhIHF1YWxpdGF0aXZlIHJlc3BvbnNlIGlzCmNvbnNpZGVyZWQgY2xhc3NpZmljYXRpb24gYmVjYXVzZSB3ZSBhc3NpZ24gdGhlIG9ic2VydmF0aW9uIHRvIGEKY2F0ZWdvcnkgKG9yIGNsYXNzKS4KCkxpa2UgcmVncmVzc2lvbiwgY2xhc3NpZmljYXRpb24gaXMgYWxzbyBhIHN1cGVydmlzZWQgbGVhcm5pbmcgdGVjaG5pcXVlCmJlY2F1c2Ugd2UgaGF2ZSBhIHNldCBvZiBsYWJlbGVkIHRyYWluaW5nIGRhdGEgdGhhdCB3ZSBjYW4gdXNlIHRvIGJ1aWxkCmEgY2xhc3NpZmllci4gTm93LCBob3dldmVyLCB3ZSBoYXZlIGFjY2VzcyB0byBkaWZmZXJlbnQgdGVjaG5pcXVlcwpiZWNhdXNlIHRoZSBzdHJ1Y3R1cmUgb2Ygb3VyIG91dGNvbWUgdmFyaWFibGUgaXMgY2F0ZWdvcmljYWwuCgojIyDwn5OdIFBvbGwgMTogQ2xhc3NpZmljYXRpb24gdnMuIFJlZ3Jlc3Npb24KCklkZW50aWZ5IHdoaWNoIG9mIHRoZSBmb2xsb3dpbmcgYXJlIGNsYXNzaWZpY2F0aW9uIHByb2JsZW1zIGluIG1hY2hpbmUKbGVhcm5pbmcuCgoxLiAgQW4gYWR2ZXJ0aXNlciBpcyBpbnRlcmVzdGVkIGluIHRoZSByZWxhdGlvbnNoaXAgYmV0d2VlbiBhZ2UgYW5kIHRoZQogICAgbnVtYmVyIG9mIGhvdXJzIG9mIFlvdVR1YmUgY29uc3VtZWQuCgoyLiAgQSBtZWRpY2FsIHRlc3RpbmcgY29tcGFueSBjb25kdWN0cyBhIHByb2NlZHVyZSB0byBkZXRlcm1pbmUgd2hldGhlcgogICAgYSBwZXJzb24gaGFzIGEgY2FuY2VyIGRpYWdub3Npcy4KCjMuICBBIHJlc2VhcmNoZXIgaXMgaW50ZXJlc3RlZCBpbiB0aGUgZWZmZWN0IG9mIGFuIGVkdWNhdGlvbgogICAgaW50ZXJ2ZW50aW9uIG9uIHN0dWRlbnRzJyB0ZXN0IHNjb3Jlcy4KCjQuICBBIHNvZnR3YXJlIGVuZ2luZWVyIGlzIGRlc2lnbmluZyBhbiBhbGdvcml0aG0gdG8gZGV0ZWN0IHdoZXRoZXIgYW4KICAgIGVtYWlsIGlzIHNwYW0gb3Igbm90LgoKNS4gIEEgcG9saXRpY2FsIHNjaWVudGlzdCB3YW50cyB0byBjbGFzc2lmeSBUd2l0dGVyIHBvc3RzIGFzIHBvc2l0aXZlIG9yCiAgICBuZWdhdGl2ZS4KCioqU29sdXRpb24gMTogKDIpLCAoNCksIGFuZCAoNSkuKioKCiMgVG9kYXkncyBEYXRhOiBWb3RpbmcgUGF0dGVybnMgaW4gdGhlIDIwMjAgRWxlY3Rpb24KCk5vdywgbGV0J3MgbG9hZCBvdXIgcHJpbWFyeSBkYXRhIHNldCBmb3IgdG9kYXkncyB3b3Jrc2hvcDogYHZvdGUyMDIwYC4KT3VyIGdvYWwgaXMgZ29pbmcgdG8gYmUgcHJlZGljdGluZyB3aGV0aGVyIHNvbWVvbmUgdm90ZWQgaW4gdGhlIDIwMjAKZWxlY3Rpb24sIHBlcmhhcHMgdG8gdGFpbG9yIGVuZ2FnZW1lbnQgc3RyYXRlZ2llcyB0b3dhcmQgdGhvc2UgbGVhc3QKbGlrZWx5IHRvIHZvdGUgaW4gYW4gdXBjb21pbmcgZWxlY3Rpb24uIFRoZXNlIGRhdGEgYXJlIGJhc2VkIG9uIHRoZSBDdXJyZW50IFBvcHVsYXRpb24gU3VydmV5J3MgW1ZvdGVyIFN1cHBsZW1lbnRdKGh0dHBzOi8vY3BzLmlwdW1zLm9yZy9jcHMvdm90ZXJfc2FtcGxlX25vdGVzLnNodG1sKS4gCgpgYGB7cn0Kdm90ZTIwMjAgPC0gcmVhZC5jc3YoIi4uL2RhdGEvdm90ZTIwMjAuY3N2Iixyb3cubmFtZXMgPSBOVUxMKQoKIyB2aXN1YWxseSBpbnNwZWN0IHRoZSBkYXRhIGZyYW1lIAp2aWV3KHZvdGUyMDIwKQpgYGAKCiMgRXhwbG9yYXRvcnkgQW5hbHlzaXMKClJlY2FsbCB0aGF0IHRoZSBmaXJzdCBzdGVwIGluIG1hY2hpbmUgbGVhcm5pbmcgKGFuZCBtb3N0IGFuYWx5c2VzKSBpcyB0bwpleHBsb3JlIHRoZSBkYXRhIHdlJ3JlIHdvcmtpbmcgd2l0aCB0byBnZXQgYSBzZW5zZSBvZiBpdHMgc2hhcGUgYW5kCmFudGljaXBhdGUgcHJvYmxlbXMgdGhhdCBtYXkgYXJpc2UuCgojIyDwn6WKIENoYWxsZW5nZSAxOiBFeHBsb3JhdG9yeSBEYXRhIEFuYWx5c2lzCgpQZXJmb3JtIGV4cGxvcmF0b3J5IGFuYWx5c2VzIG9uIHRoZSBgdm90ZTIwMjBgIGRhdGEgc2V0LCBrZWVwaW5nIGluIG1pbmQKdGhhdCB0b2RheSdzIGdvYWwgaXMgdG8gcHJlZGljdCBgdm90ZWRgLiBXaGF0IGRvIHlvdSBub3RpY2UgYWJvdXQgdGhlCmRhdGE/CgpgYGB7cn0KIyBjcmVhdGUgYmFyIGNoYXJ0IG9mIHZvdGluZyBwZXJjZW50YWdlcyBieSBzdGF0ZSAKdm90ZTIwMjAgJT4lCiAgZ3JvdXBfYnkoc3RhdGUpICU+JQogIHN1bW1hcmlzZSh2b3RlZF9wZXJjZW50YWdlID0gbWVhbih2b3RlZCwgbmEucm0gPSBUUlVFKSAqIDEwMCkgJT4lCiAgdW5ncm91cCgpICU+JQogIGdncGxvdChhZXMoeCA9IHJlb3JkZXIoc3RhdGUsLXZvdGVkX3BlcmNlbnRhZ2UpLCB5ID0gdm90ZWRfcGVyY2VudGFnZSkpICsKICAgIGdlb21fYmFyKHN0YXQgPSAiaWRlbnRpdHkiLCBmaWxsID0gImJsdWUiKSArCiAgICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCB2anVzdCA9IDAuNSwgaGp1c3QgPSAxKSkgKyAgIyBBZGp1c3RpbmcgeC1heGlzIHRleHQgKwogIGNvb3JkX2ZsaXAoKSArIAogIHNjYWxlX3lfY29udGludW91cyhsaW1pdHM9YygwLDEwMCkpICsgCiAgICBsYWJzKHRpdGxlID0gIlBlcmNlbnRhZ2Ugb2YgUGVvcGxlIFdobyBWb3RlZCBpbiBFYWNoIFN0YXRlIiwKICAgICAgICAgeCA9ICJTdGF0ZSIsCiAgICAgICAgIHkgPSAiUGVyY2VudGFnZSBWb3RlZCIpICsKICAgIHRoZW1lX21pbmltYWwoKQpgYGAKCiMgVHJhaW4tVGVzdCBTcGxpdAoKUmVjYWxsIGZyb20gUGFydCAxIHRoYXQgYSBjcml0aWNhbCBmaXJzdCBzdGVwIGluIG1hY2hpbmUgbGVhcm5pbmcgaXMKcGFydGl0aW9uaW5nIG91ciBkYXRhIGludG8gdHJhaW5pbmcgYW5kIHRlc3Qgc2V0cy4KCiMjIPCflJQgUXVlc3Rpb24gMTogVHJhaW4tVGVzdCBTcGxpdAoKQSByZXNlYXJjaGVyIHVzZWQgdGhlIGVudGlyZSBgdm90ZTIwMjBgIGRhdGEgc2V0IHRvIHRyYWluIGEgbW9kZWwgdGhhdApyZXN1bHRlZCBpbiBpbXByZXNzaXZlbHkgaGlnaCBhY2N1cmFjeSBkdXJpbmcgZXZhbHVhdGlvbi4gSG93ZXZlciwgdXBvbgpkZXBsb3lpbmcgdGhpcyBtb2RlbCB0byBwcmVkaWN0IHZvdGVyIHR1cm5vdXQgZm9yIGFuIHVwY29taW5nIGVsZWN0aW9uLAp0aGUgcHJlZGljdGlvbnMgc2lnbmlmaWNhbnRseSBkaXZlcmdlZCBmcm9tIGFjdHVhbCB0dXJub3V0LCB3aXRoIG1hbnkKZGlzY3JlcGFuY2llcyBpbiB3aG8gd2FzIHByZWRpY3RlZCB0byB2b3RlIHZlcnN1cyB3aG8gYWN0dWFsbHkgdm90ZWQuClRoZSBtb2RlbCdzIG5lYXItcGVyZmVjdCBwZXJmb3JtYW5jZSBpbiBkZXZlbG9wbWVudCBzdGFya2x5IGNvbnRyYXN0ZWQKaXRzIHBvb3IgcmVhbC13b3JsZCBwcmVkaWN0aW9uIG91dGNvbWVzLCBpbmRpY2F0aW5nIGEgcG90ZW50aWFsCm92ZXJzaWdodCBpbiB0aGUgbW9kZWwgdHJhaW5pbmcgYW5kIGV2YWx1YXRpb24gcHJvY2Vzcy4gV2h5IG1pZ2h0IHRoYXQKYmU/CgoqKkFuc3dlciAxKio6IFRoZSBrZXkgaXNzdWUgaXMgdGhhdCB0aGUgbW9kZWwgd2FzIG5ldmVyIHRlc3RlZCBvbiB1bnNlZW4KZGF0YS4gQnkgdXNpbmcgdGhlIGVudGlyZSBkYXRhIHNldCBmb3IgdHJhaW5pbmcsIHRoZSBtb2RlbCBlc3NlbnRpYWxseQoibWVtb3JpemVkIiB0aGUgZGF0YSwgaW5jbHVkaW5nIGFueSBub2lzZSBvciBwYXR0ZXJucyBzcGVjaWZpYyB0byB0aGF0CnNldCBvZiBpbmRpdmlkdWFscy4gVGhpcyBsZWQgdG8gb3ZlcmZpdHRpbmcsIHdoZXJlIHRoZSBtb2RlbCBleGNlbGxlZCBhdApwcmVkaWN0aW5nIHRoZSB0cmFpbmluZyBkYXRhIGJ1dCBmYWlsZWQgdG8gZ2VuZXJhbGl6ZSB0byBuZXcsIHVuc2VlbgpkYXRhLiBTaW5jZSB0aGUgbW9kZWwncyBwZXJmb3JtYW5jZSB3YXMgb25seSBldmFsdWF0ZWQgb24gZGF0YSBpdCBoYWQKYWxyZWFkeSBzZWVuLCBpdHMgYXBwYXJlbnQgYWNjdXJhY3kgd2FzIG1pc2xlYWRpbmcsIGdpdmluZyBhIGZhbHNlIHNlbnNlCm9mIGNvbmZpZGVuY2UgaW4gaXRzIHByZWRpY3RpdmUgY2FwYWJpbGl0aWVzLgoKTm93IHRoYXQgd2UgaGF2ZSBhIGJldHRlciBzZW5zZSBvZiB0aGUgZGF0YSwgbGV0J3MgZ28gYWhlYWQgYW5kIHNwbGl0CmB2b3RlMjAyMGAgaW50byB0cmFpbmluZyBhbmQgdGVzdCBzZXRzOgoKYGBge3J9CiMgUGVyZm9ybSBzcGxpdHMKdm90ZTIwMjAkdm90ZWQgPC0gYXMuZmFjdG9yKHZvdGUyMDIwJHZvdGVkKQp2b3RlX3NwbGl0IDwtIGluaXRpYWxfc3BsaXQodm90ZTIwMjAsIHByb3AgPSAwLjc1KQp2b3RlX3RyYWluIDwtIHRyYWluaW5nKHZvdGVfc3BsaXQpCnZvdGVfdGVzdCA8LSB0ZXN0aW5nKHZvdGVfc3BsaXQpCmBgYAoKIyBQcmUtUHJvY2Vzc2luZwoKSW4gb3VyIGV4YW1wbGUgZnJvbSBQYXJ0IDEsIHdlIHByZXBhcmVkIG91ciBkYXRhIGZvciBhbmFseXNpcyBieQpyZWNvZGluZyBjYXRlZ29yaWNhbCB2YXJpYWJsZXMgYW5kIG5vcm1hbGl6aW5nIG51bWVyaWMgb25lcyB1c2luZwpgc3RlcF9kdW1teShhbGxfbm9taW5hbF9wcmVkaWN0b3JzKCkpYCBhbmQgYHN0ZXBfbm9ybWFsaXplKClgLApyZXNwZWN0aXZlbHkuCgpJbiB0aGF0IGV4YW1wbGUsIHdlIGFsc28gZHJvcHBlZCByb3dzIHRoYXQgaGFkIGFueSBtaXNzaW5nIHZhbHVlcyBhY3Jvc3MKdmFyaWFibGVzLiBMZXQncyB0cnkgYW5vdGhlciBleGFtcGxlLCBpbiB3aGljaCB3ZSBkb24ndCAqb21pdCogc2FtcGxlcwp0aGF0IGhhdmUgbWlzc2luZyB2YWx1ZXMsIGJ1dCBpbnN0ZWFkIHBlcmZvcm0gKmltcHV0YXRpb24qLCBpbiB3aGljaCB3ZQpyZXBsYWNlIHRob3NlIG1pc3NpbmcgdmFsdWVzIGFjY29yZGluZyB0byBjZXJ0YWluIGNyaXRlcmlhLiBUaGVyZSBhcmUKdmFyaW91cyBraW5kcyBvZiBpbXB1dGF0aW9uLCBpbmNsdWRpbmc6CgotICAgRm9yIGV4YW1wbGUsIHdoZW5ldmVyIHdlIGhhdmUgYSBtaXNzaW5nIHZhbHVlIGZvciB0aGUgYG9jY3VwYXRpb25gLAogICAgd2UgY2FuIHJlcGxhY2UgdGhhdCBtaXNzaW5nIHZhbHVlIHdpdGggdGhlIG1vc3QgY29tbW9uIG9jY3VwYXRpb24uCiAgICBUaGlzIGlzIGNhbGxlZCAqbW9kZSBpbXB1dGF0aW9uKi4KCi0gICBPciwgd2UgY291bGQgcmVwbGFjZSBhIG1pc3NpbmcgbnVtZXJpY2FsIHByZWRpY3RvciAoZS5nLiwgYWdlKSB1c2luZwogICAgdGhlIG1lZGlhbiBhY3Jvc3MgYWxsIHRoZSBzYW1wbGVzLiBUaGlzIGlzIGNhbGxlZCAqbWVkaWFuCiAgICBpbXB1dGF0aW9uKi4KClRoZXJlIGFyZSBvdGhlciB3YXlzIHRvIGltcHV0ZSwgYnV0IHRoZXNlIGFyZSBnb29kIHN0YXJ0aW5nIHBvaW50cy4gVGhlCndheSB0byBwZXJmb3JtIGltcHV0YXRpb24gaW4gYSBgcmVjaXBlYCBpcyB2aWEgdGhlIGBzdGVwX2ltcHV0ZV9gCmZ1bmN0aW9ucy4KCiMjIPCfpYogQ2hhbGxlbmdlIDI6IENyZWF0aW5nIGEgUmVjaXBlCgpVc2luZyB0aGUgc2FtZSBsb2dpYyBmcm9tIFBhcnQgMSwgY3JlYXRlIHlvdXIgb3duIHJlY2lwZSBjYWxsZWQKYHZvdGVyZWNpcGVgIHRoYXQgbGF5cyBvdXQgdGhlIHN0ZXBzIGZvciBwcmUtcHJvY2Vzc2luZyB0aGUgZGF0YS4KSW5zdGVhZCBvZiBkcm9wcGluZyByb3dzIHdpdGggbWlzc2luZyB2YWx1ZXMsIHVzZQpbc3RlcF9pbXB1dGVfbWVkaWFuXShodHRwczovL3JlY2lwZXMudGlkeW1vZGVscy5vcmcvcmVmZXJlbmNlL3N0ZXBfaW1wdXRlX21lZGlhbi5odG1sKQp0byBpbXB1dGUgbnVtZXJpYyB2YXJpYWJsZXMgYW5kCltzdGVwX2ltcHV0ZV9tb2RlXShodHRwczovL3JlY2lwZXMudGlkeW1vZGVscy5vcmcvcmVmZXJlbmNlL3N0ZXBfaW1wdXRlX21vZGUuaHRtbCkKdG8gaW1wdXRlIGNhdGVnb3JpY2FsIHZhcmlhYmxlcy4KCmBgYHtyfQp2b3RlcmVjaXBlIDwtIAogIHJlY2lwZSh2b3RlZCB+IC4sIGRhdGEgPSB2b3RlX3RyYWluKSAlPiUKICBzdGVwX2ltcHV0ZV9tZWRpYW4oYWxsX251bWVyaWNfcHJlZGljdG9ycygpKSAlPiUKICBzdGVwX2ltcHV0ZV9tb2RlKGFsbF9ub21pbmFsX3ByZWRpY3RvcnMoKSkgJT4lCiAgc3RlcF9kdW1teShhbGxfbm9taW5hbF9wcmVkaWN0b3JzKCkpIApgYGAKClRoaXMgcHJlLXByb2Nlc3Npbmcgd2lsbCBhbGxvdyB1cyB0byB0YWtlIGFkdmFudGFnZSBvZiBzYW1wbGVzIHdpdGgKbWlzc2luZyBkYXRhLCBldmVuIGlmIGl0IGNvbWVzIGF0IGEgbGl0dGxlIGNvc3QgdG8gYWNjdXJhY3kuIEltcHV0YXRpb24KaXMgb2Z0ZW4gYSBuZWNlc3Nhcnkgc3RlcCwgc2luY2UgaXQncyBjb21tb24gdG8gaGF2ZSBtaXNzaW5nIGRhdGEuCgojIyDwn5SUIFF1ZXN0aW9uIDI6IEltcHV0YXRpb24KCk5vdGljZSB0aGF0IHdlIGhhdmUgb25seSBhcHBsaWVkIG91ciByZWNpcGUgdG8gdGhlIHRyYWluaW5nIGRhdGEgc28gZmFyLgpFeHBsYWluIHdoeSBpdCBpcyBpbXBvcnRhbnQgdG8gcGVyZm9ybSBpbXB1dGF0aW9uIHNlcGFyYXRlbHkgb24gdGhlCnRyYWluaW5nIGFuZCB0ZXN0IHNldHMgcmF0aGVyIHRoYW4gaW1wdXRpbmcgbWlzc2luZyB2YWx1ZXMgYmVmb3JlCnNwbGl0dGluZyB0aGUgZGF0YSBzZXQuIENvbnNpZGVyIHdoeSB3ZSBkb24ndCB3YW50IHRoZSBkYXRhIHdlIHRyYWluIG91cgptb2RlbCBvbiB0byBiZSBleHBvc2VkIHRvIG91ciB0ZXN0IGRhdGEgc2V0IGluIHRoZSBmaXJzdCBwbGFjZS4KCioqQW5zd2VyIDIqKjogV2Ugd2FudCB0byBrZWVwIG91ciB0cmFpbmluZyBhbmQgdGVzdCBzZXRzIHNlcGFyYXRlLiBJZiB3ZQppbXB1dGUgbWlzc2luZyB2YWx1ZXMgaW4gdGhlIGVudGlyZSBkYXRhIHNldCB1c2luZyB0aGUgbWVhbiwgdGhlIG1lYW4KY2FsY3VsYXRpb24gd2lsbCBpbmNsdWRlIGluZm9ybWF0aW9uIGZyb20gYm90aCB0aGUgdHJhaW5pbmcgYW5kIHRlc3QKc2V0cy4gVGhpcyBtZWFucyB0aGUgbW9kZWwgZ2V0cyBpbmRpcmVjdGx5IGV4cG9zZWQgdG8gaW5mb3JtYXRpb24gZnJvbQp0aGUgdGVzdCBzZXQgZHVyaW5nIHRyYWluaW5nLCB3aGljaCBjYW4gbGVhZCB0byBvdmVybHkgb3B0aW1pc3RpYwpwZXJmb3JtYW5jZSBlc3RpbWF0ZXMgYW5kIGEgbW9kZWwgdGhhdCBtYXkgbm90IHBlcmZvcm0gYXMgd2VsbCBvbiB0cnVseQp1bnNlZW4gZGF0YS4gVGhpcyBjb25jZXB0IGlzIHJlZmVycmVkIHRvIGFzICdkYXRhIGxlYWthZ2UuJwoKIyBEZXZlbG9waW5nIGFuZCBFdmFsdWF0aW5nIE1vZGVscwoKTm93IHRoYXQgd2UndmUgcGVyZm9ybWVkIGV4cGxvcmF0b3J5IGFuYWx5c2VzLCBhIHRyYWluLXRlc3Qgc3BsaXQsIGFuZApwcmUtcHJvY2Vzc2luZywgbGV0J3MgZ28gYWhlYWQgYW5kIHRyYWluIG91ciBtb2RlbC4gV2Ugd2lsbCBjcmVhdGUgYQpjbGFzc2lmaWVyIChpLmUuLCBhIG1vZGVsIHRoYXQgcHJlZGljdHMgbWVtYmVyc2hpcCBpbiBhIGdyb3VwKSB3aXRoCmxvZ2lzdGljIHJlZ3Jlc3Npb24uCgojIyMjIEFsZ29yaXRobSAxOiBMb2dpc3RpYyBSZWdyZXNzaW9uCgpNYWNoaW5lIGxlYXJuaW5nIHByYWN0aXRpb25lcnMgb2Z0ZW4gcmVjb21tZW5kIGxvZ2lzdGljIHJlZ3Jlc3Npb24gYXMgYQpzdGFydGluZyBtb2RlbCB3aGVuIHByZWRpY3RpbmcgYSBiaW5hcnkgb3V0Y29tZSBvciBwcm9iYWJpbGl0eS4gRm9yCmV4YW1wbGUsIGlmIHdlIGFyZSBlc3RpbWF0aW5nIHRoZSByZWxhdGlvbnNoaXAgYmV0d2VlbiBtb3J0YWxpdHkgYW5kCmluY29tZSwgd2UgY2FuIGVzdGltYXRlIHRoZSBwcm9iYWJpbGl0eSBvZiBtb3J0YWxpdHkgZ2l2ZW4gYSBjaGFuZ2UgaW4KaW5jb21lLgoKV2Ugd3JpdGUgdGhpcyBhcyAkUFtNfEldJCBhbmQgdGhlIHZhbHVlcyB3aWxsIHJhbmdlIGJldHdlZW4gMCBhbmQgMS4gV2UKY2FuIG1ha2UgYSBwcmVkaWN0aW9uIGZvciBhbnkgZ2l2ZW4gdmFsdWUgb2YgaW5jb21lIG9uIHRoZSBtb3J0YWxpdHkKb3V0Y29tZS4gTm9ybWFsbHksIHdlIGVzdGFibGlzaCBhIHRocmVzaG9sZCBmb3IgcHJlZGljdGlvbi4gRm9yIGV4YW1wbGUsCndlIG1pZ2h0IHByZWRpY3QgZGVhdGggKE0pIHdoZXJlICRQW018aV0gPiAwLjUkLgoKTG9naXN0aWMgcmVncmVzc2lvbiBpcyBhIGdlbmVyYWxpemVkIGxpbmVhciBtb2RlbCB3aGVyZSB3ZSBtb2RlbCB0aGUKcHJvYmFiaWxpdHkgZnVuY3Rpb24gdXNpbmcgdGhlIGxvZ2lzdGljIGZ1bmN0aW9uLCBhbiBzLXNoYXBlZCBjdXJ2ZQpzb21ldGltZXMgY2FsbGVkIGEgc2lnbW9pZCBmdW5jdGlvbi4gVGhlIGdlbmVyYWwgZnVuY3Rpb24gaXM6CiRwKFk9IDF8WCkkLiBJbiB0aGUgZXhhbXBsZSBiZWxvdywgdGhlIGdyZWVuIGRhdGEgcG9pbnRzIHdpbGwgYmUKY2xhc3NpZmllZCBhcyAwIGJlY2F1c2UgJHAkIFw8IDAuNTsgdGhlIG9yYW5nZSBwb2ludHMgd2lsbCBiZSBjbGFzc2lmaWVkCmFzIDEgYmVjYXVzZSAkcCQgXD4gMC41LgoKIVtMb2dpc3RpYyBSZWdyZXNzaW9uXSguLi9pbWFnZXMvTG9naXN0aWNfUmVncmVzc2lvbi5wbmcpCgpIZXJlJ3MgaG93IHRvIGZpdCBjbGFzc2lmaWNhdGlvbiBwcm9ibGVtcyBpbiBgdGlkeW1vZGVsc2AgdXNpbmcgbG9naXN0aWMKcmVncmVzc2lvbi4KCkluIGB0aWR5bW9kZWxzYCwgY3JlYXRpbmcgYSBsb2dpc3RpYyByZWdyZXNzaW9uIGZvbGxvd3MgdGhlIGV4YWN0IHNhbWUKcHJvY2VkdXJlIGFzIGEgbGluZWFyIHJlZ3Jlc3Npb24uIFRoaXMgdGltZSwgaG93ZXZlciwgd2Ugd2lsbCB1c2UKYGxvZ2lzdGljX3JlZygpYCB0byBpbml0aWF0ZSB0aGUgZnVuY3Rpb24uIExldCdzIGNyZWF0ZSB0aGUgbW9kZWw6CgpgYGB7cn0KIyBDcmVhdGUgbW9kZWwKbG9naXN0aWNfbW9kZWwgPC0gbG9naXN0aWNfcmVnKG1vZGUgPSAiY2xhc3NpZmljYXRpb24iKQpgYGAKCk5vdywgbGV0J3MgZml0IHRoZSBtb2RlbCBvbiB0aGUgdHJhaW5pbmcgZGF0YS4gV2UgZmlyc3QgY3JlYXRlIGEgd29ya2Zsb3cgdGhhdCBsaXN0cyBpbnN0cnVjdGlvbnMgZm9yIHByZS1wcm9jZXNzaW5nIGFuZCB0aGUgbW9kZWwgd2Ugd2FudCB0byB1c2UsIGFuZCB0aGVuIGFwcGx5IGl0IHRvIHRoZSB0cmFpbmluZyBkYXRhLiAKCmBgYHtyfQp2b3RlX3dmbG93IDwtIHdvcmtmbG93KCkgJT4lCiAgYWRkX3JlY2lwZSh2b3RlcmVjaXBlKSAlPiUKICBhZGRfbW9kZWwobG9naXN0aWNfbW9kZWwpCgp2b3RlX2ZpdCA8LSBmaXQodm90ZV93Zmxvdywgdm90ZV90cmFpbikKdm90ZV9maXQgJT4lIHRpZHkoKQpgYGAKCkZpbmFsbHksIGxldCdzIHVzZSBhdWdtZW50IGZyb20gdGhlIGB5YXJkc3RpY2tgIHBhY2thZ2UgdG8gb2J0YWluIHRoZQpwcmVkaWN0aW9ucywgYW5kIHRha2UgYSBsb29rIGF0IHRoZW06CgpgYGB7cn0KbG9naXN0aWNfcHJlZGljdGlvbnMgPC0gYXVnbWVudCh2b3RlX2ZpdCwgbmV3X2RhdGEgPSB2b3RlX3Rlc3QpCmxvZ2lzdGljX3ByZWRpY3Rpb25zWywxNDoxN10gIyBsb29rIGF0IGxhc3QgNCBjb2x1bW5zIApgYGAKCiMjIPCflJQgUXVlc3Rpb24gMzogQW5hbHl6aW5nIE91dHB1dAoKTm90aWNlIHRoYXQgZm91ciBuZXcgY29sdW1ucyBoYXZlIGFwcGVhcmVkIGluIG91ciBkYXRhIHNldC4gV2hhdCBkbwp0aGVzZSB2YWx1ZXMgbWVhbj8gV2hhdCBhcmUgdGhleSB0ZWxsaW5nIHVzPwoKKipBbnN3ZXIgMyoqOiBgLnByZWRfY2xhc3NgIGNvcnJlc3BvbmRzIHRvIHRoZSB2YWx1ZSBvdXIgYWxnb3JpdGhtIGlzCnByZWRpY3RpbmcsIGkuZS4sIHdoZXRoZXIgdGhlIHBlcnNvbiB2b3RlZCBpbiB0aGUgZWxlY3Rpb24gb3Igbm90LiBUaGlzCndhcyBkZXRlcm1pbmVkIHVzaW5nIHRoZSB0d28gdmFyaWFibGVzIGAucHJlZF8wYCBhbmQgYC5wcmVkXzFgLCB3aGljaApyZXByZXNlbnQgdGhlIHByZWRpY3Rpb25zIGZyb20gdGhlIGxvZ2lzdGljIHJlZ3Jlc3Npb24uIElmIG1lbWJlcnNoaXAgaW4KZ3JvdXAgMSAoaS5lLiwgdm90ZWQgPSAxKSBpcyBncmVhdGVyIHRoYW4gLjUsIHRoZW4gYC5wcmVkX2NsYXNzYCB3aWxsCnRha2UgYSB2YWx1ZSBvZiAxLgoKVG8gZXZhbHVhdGUgdGhlIG1vZGVsLCB3ZSdsbCB1c2UgdGhlIGBhY2N1cmFjeWAgZnVuY3Rpb246CgpgYGB7cn0KYWNjdXJhY3kobG9naXN0aWNfcHJlZGljdGlvbnMsIHRydXRoID0gdm90ZWQsIGVzdGltYXRlID0gLnByZWRfY2xhc3MsIHR5cGU9ImNsYXNzIikKYGBgCgpXZSBwcmVkaWN0ZWQgdGhlIGxpa2VsaWhvb2QgdGhhdCBzb21lb25lIHZvdGVkIGluIHRoZSBsYXN0IGVsZWN0aW9uIHdpdGgKYW4gYWNjdXJhY3kgb2YgYWJvdXQgODElIC0gbm90IGJhZCEKCldlIGNhbiBhbHNvIGNyZWF0ZSB3aGF0J3MgY2FsbGVkIGEgJ2NvbmZ1c2lvbiBtYXRyaXgnIHRvIHNlZSBob3cgd2VsbApvdXIgbW9kZWwgZGlkIGF0IHByZWRpY3RpbmcgZWFjaCBjbGFzcy4gQSBjb25mdXNpb24gbWF0cml4IGlzIGhlbHBmdWwgaWYKeW91IGNhcmUgYWJvdXQgZmFsc2UgcG9zaXRpdmVzIChwcmVkaWN0aW5nIDEgd2hlbiB0aGUgdHJ1ZSB2YWx1ZSBpcyAwKQphbmQgZmFsc2UgbmVnYXRpdmVzIChwcmVkaWN0aW5nIDAgd2hlbiB0aGUgdHJ1ZSB2YWx1ZSBpcyAxKS4gSW4gdGhlCmNvbnRleHQgb2YgdGVzdGluZyBmb3IgQ292aWQtMTksIHdlIG1pZ2h0IHdhbnQgY2FyZSBtb3JlIGFib3V0IGZhaWxpbmcKdG8gZGlhZ25vc2UgYSBwZXJzb24gd2l0aCB0aGUgdmlydXMgKGkuZS4sIGEgZmFsc2UgbmVnYXRpdmUpIGJlY2F1c2UgdGhlCnN0YWtlcyBhcmUgaGlnaGVyLiAKCmBgYHtyfQpsb2dpc3RpY19wcmVkaWN0aW9ucyAlPiUKICBjb25mX21hdCh0cnV0aCA9IHZvdGVkLCBlc3RpbWF0ZSA9IC5wcmVkX2NsYXNzKQpgYGAKClNvIGZhciB3ZSBoYXZlIHRyYWluZWQgYSBzaW1wbGUgbG9naXN0aWMgcmVncmVzc2lvbiBtb2RlbCwgdGhlcmUgYXJlIG1hbnkgb3RoZXIgb3B0aW9ucyBpbiBvdXIgbWFjaGluZSBsZWFybmluZyBhcnNlbmFsIHRvIGdldCBhIGJldHRlciBwcmVkaWN0aW9uLgoKIyMjIyBIeXBlcnBhcmFtZXRlciBUdW5pbmcgYW5kIFJlZ3VsYXJpemF0aW9uCgpOb3RpY2UgdGhhdCB1bnRpbCBub3csIHdlIGhhdmUgdXNlZCB0aGUgZGVmYXVsdCBzZXR0aW5ncyBmb3IgdGhlIGxpbmVhcgphbmQgbG9naXN0aWMgcmVncmVzc2lvbnMgd2UgaGF2ZSBydW4uIExvb2tpbmcgYXQgdGhlCihkb2N1bWVudGF0aW9uKVs8aHR0cHM6Ly93d3cucmRvY3VtZW50YXRpb24ub3JnL3BhY2thZ2VzL3BhcnNuaXAvdmVyc2lvbnMvMC4wLjAuOTAwMS90b3BpY3MvbG9naXN0aWNfcmVnPl0KZm9yIGxvZ2lzdGljIHJlZ3Jlc3Npb24sIHdlIHNlZSB0aGF0IHRoZXJlIGFyZSBtYW55IGFyZ3VtZW50cyB0aGF0IHdlCmxlZnQgYmxhbmsgd2hlbiB3ZSBpbml0aWFsaXplZCBvdXIgbW9kZWwgYWJvdmUuIFRoZXNlIGFyZ3VtZW50cywgYWxzbwprbm93biBhcyAncGFyYW1ldGVycycsIGNvcnJlc3BvbmQgdG8gc3RhdGlzdGljYWwgY2hvaWNlcyBhYm91dCBob3cgdGhlCm1vZGVsIHNob3VsZCBvcGVyYXRlIG9yIGJlIHN0cnVjdHVyZWQuCgpTb21lIG9mIHRoZXNlIGh5cGVycGFyYW1ldGVycyBpbmNsdWRlICdlbmdpbmUnIGFuZCAncGVuYWx0eScuCgotICAgRW5naW5lOiBEaWZmZXJlbnQgZW5naW5lcyBjYW4gaW1wbGVtZW50IHJlZ3Jlc3Npb24gdGhyb3VnaCB2YXJpb3VzCiAgICBhbGdvcml0aG1zIG9yIGNvbXB1dGF0aW9uYWwgYXBwcm9hY2hlcy4gV2hlbiB5b3Ugc3BlY2lmeSBhbiBlbmdpbmUKICAgIGZvciBsb2dpc3RpYyByZWdyZXNzaW9uIGluIFIsIHlvdSdyZSBjaG9vc2luZyB0aGUgcGFydGljdWxhciBzZXQgb2YKICAgIGFsZ29yaXRobXMgYW5kIG9wdGltaXphdGlvbnMgdGhhdCB3aWxsIGJlIHVzZWQgdG8gdHJhaW4geW91ciBtb2RlbC4KCi0gICBQZW5hbHR5OiAicGVuYWx0eSIgcmVmZXJzIHRvIGEgcmVndWxhcml6YXRpb24gdGVjaG5pcXVlIHVzZWQgdG8KICAgIHByZXZlbnQgb3ZlcmZpdHRpbmcgYnkgZGlzY291cmFnaW5nIG92ZXJseSBjb21wbGV4IG1vZGVscy4gSXQgZG9lcwogICAgdGhpcyBieSBhZGRpbmcgYSBwZW5hbHR5IHRvIHRoZSBsb3NzIGZ1bmN0aW9uIGZvciBsYXJnZQogICAgY29lZmZpY2llbnRzLiBDb21tb24gcGVuYWx0aWVzIGFyZSBMMSAoTGFzc28pLCB3aGljaCBjYW4gc2hyaW5rIHNvbWUKICAgIGNvZWZmaWNpZW50cyB0byB6ZXJvICh0aHVzIHBlcmZvcm1pbmcgZmVhdHVyZSBzZWxlY3Rpb24pLCBhbmQgTDIKICAgIChSaWRnZSksIHdoaWNoIHNocmlua3MgYWxsIGNvZWZmaWNpZW50cyB0b3dhcmQgemVybyBidXQgdHlwaWNhbGx5CiAgICBkb2Vzbid0IHNldCBhbnkgdG8gZXhhY3RseSB6ZXJvLiBUaGUgcGVuYWx0eSBoZWxwcyBpbiBjcmVhdGluZwogICAgc2ltcGxlciwgbW9yZSBnZW5lcmFsaXphYmxlIG1vZGVscyB0aGF0IHBlcmZvcm0gYmV0dGVyIG9uIHVuc2VlbgogICAgZGF0YSBieSBwcmlvcml0aXppbmcgdGhlIG1vc3QgaW5mbHVlbnRpYWwgZmVhdHVyZXMgYW5kIHJlZHVjaW5nIHRoZQogICAgbW9kZWwncyBzZW5zaXRpdml0eSB0byB0aGUgdHJhaW5pbmcgZGF0YSdzIG5vaXNlLgoKRm9yIGV4YW1wbGUsIHdlIGNhbiBhZGQgYSAncGVuYWx0eScgb24gdGhlIHNpemUgb2YgdGhlIGNvZWZmaWNpZW50cyBvZiBhCm1vZGVsIGFuZCByZWR1Y2UgdGhlIGxpa2VsaWhvb2Qgb2Ygb3ZlcmZpdHRpbmcuIExhc3NvLCByaWRnZSwgYW5kCmVsYXN0aWMgbmV0IGFyZSBkaWZmZXJlbnQgdHlwZXMgb2YgcGVuYWx0aWVzIHRoYXQgZ3JlYXRseSByZWR1Y2Ugb3IKc2hyaW5rIHRvIHplcm8gY29lZmZpY2llbnRzIG9uIHZhcmlhYmxlcyB0aGF0IGFyZSBwaWNraW5nIHVwIG9uIGEgbG90IG9mCm5vaXNlLiBUaGUgYnJvYWQgdGVjaG5pcXVlIG9mIHJlZHVjaW5nIG92ZXJmaXR0aW5nIGlzIGNhbGxlZAoncmVndWxhcml6YXRpb24uJwoKIyMg8J+liiBDaGFsbGVuZ2UgMzogQ2hhbmdpbmcgSHlwZXJwYXJhbWV0ZXJzCgpXZSBoYXZlIHJlcHJvZHVjZWQgdGhlIG9yaWdpbmFsIGNvZGUgZnJvbSBhYm92ZSB0aGF0IHRyYWluZWQgYSBiYXNpYwpjbGFzc2lmaWVyIG9uIG91ciB2b3RpbmcgZGF0YSBzZXQgd2l0aG91dCBjaGFuZ2luZyBhbnkgb2YgdGhlCmh5cGVycGFyYW1ldGVycywgYXMgd2VsbCBhcyB0aGUgY29kZSB0aGF0IG9idGFpbnMgcHJlZGljdGlvbnMuIFJlLXJ1bgp0aGlzIGNvZGUgc2V2ZXJhbCB0aW1lcywgYnV0IGVhY2ggdGltZSBjaGFuZ2UgdGhlIGh5cGVycGFyYW1ldGVycyBpbiB0aGUKbW9kZWwgc3BlY2lmaWNhdGlvbi4gRm9yIHBlbmFsdHksIGluY2x1ZGUgYSBub24tbmVnYXRpdmUgbnVtYmVyOyBhbmQgZm9yCmVuZ2luZSwgc2VsZWN0IGVpdGhlciAiZ2xtbmV0IiBvciAiZ2xtIi4gSG93IGRvZXMgdGhlIGFjY3VyYWN5IGNoYW5nZT8KCmBgYHtyfQpjaGFsM19tb2RlbCA8LSBsb2dpc3RpY19yZWcobW9kZSA9ICJjbGFzc2lmaWNhdGlvbiIsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZW5naW5lPSJnbG1uZXQiLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBlbmFsdHk9MCkKCmNoYWwzX3dmbG93IDwtIHdvcmtmbG93KCkgJT4lCiAgYWRkX3JlY2lwZSh2b3RlcmVjaXBlKSAlPiUKICBhZGRfbW9kZWwoY2hhbDNfbW9kZWwpCgpjaGFsM19maXQgPC0gZml0KGNoYWwzX3dmbG93LCB2b3RlX3RyYWluKQoKY2hhbDNfcHJlZGljdGlvbnMgPC0gYXVnbWVudChjaGFsM19maXQsIG5ld19kYXRhID0gdm90ZV90ZXN0KQoKYWNjdXJhY3koY2hhbDNfcHJlZGljdGlvbnMsIHRydXRoID0gdm90ZWQsIGVzdGltYXRlID0gLnByZWRfY2xhc3MsIHR5cGU9ImNsYXNzIikKYGBgCgpJdCdzIG5pY2UgdGhhdCB3ZSBjYW4gZG8gZGlmZmVyZW50IHR5cGVzIG9mIHJlZ3VsYXJpemF0aW9uLCBidXQgaG93IGRvCndlIGtub3cgd2hhdCB2YWx1ZSBvZiB0aGUgcGVuYWx0eSBjb2VmZmljaWVudCB0byBwaWNrPyBJbiBtYWNoaW5lCmxlYXJuaW5nLCB0aGlzIHZhbHVlIC0gd2hpY2ggd2UgY2hvb3NlIGJlZm9yZSBmaXR0aW5nIHRoZSBtb2RlbCAtIGlzCmtub3duIGFzIGEgaHlwZXJwYXJhbWV0ZXIuIFNpbmNlIGh5cGVycGFyYW1ldGVycyBhcmUgY2hvc2VuICpiZWZvcmUqIHdlCmZpdCB0aGUgbW9kZWwsIHdlIGNhbid0IGp1c3QgY2hvb3NlIHRoZW0gYmFzZWQgb2ZmIHRoZSB0cmFpbmluZyBkYXRhLgpTbywgaG93IHNob3VsZCB3ZSBnbyBhYm91dCBjb25kdWN0aW5nICoqaHlwZXJwYXJhbWV0ZXIgdHVuaW5nKio6CmlkZW50aWZ5aW5nIHRoZSBiZXN0IGh5cGVycGFyYW1ldGVyKHMpIHRvIHVzZT8KCkxldCdzIHRoaW5rIGJhY2sgdG8gb3VyIG9yaWdpbmFsIGdvYWwuIFdlIHdhbnQgYSBtb2RlbCB0aGF0IGdlbmVyYWxpemVzCnRvIHVuc2VlbiBkYXRhLiBTbywgaWRlYWxseSwgdGhlIGNob2ljZSBvZiB0aGUgaHlwZXJwYXJhbWV0ZXIgc2hvdWxkIGJlCnN1Y2ggdGhhdCB0aGUgcGVyZm9ybWFuY2Ugb24gdW5zZWVuIGRhdGEgaXMgdGhlIGJlc3QuIFdlIGNhbid0IHVzZSB0aGUKdGVzdCBzZXQgZm9yIHRoaXMsIGJ1dCB3aGF0IGlmIHdlIGhhZCBhbm90aGVyIHNldCBvZiBoZWxkLW91dCBkYXRhPwoKIyBIeXBlcnBhcmFtZXRlciB0dW5pbmcKCkN1ZSBoeXBlcnBhcmFtZXRlciB0dW5pbmchIEh5cGVycGFyYW1ldGVyIHR1bmluZyBpcyBjcnVjaWFsIGluIG1hY2hpbmUKbGVhcm5pbmcgYmVjYXVzZSBpdCBkaXJlY3RseSBpbXBhY3RzIHRoZSBwZXJmb3JtYW5jZSBhbmQgZWZmZWN0aXZlbmVzcwpvZiBtb2RlbHMuIEJ5IGZpbmUtdHVuaW5nIGh5cGVycGFyYW1ldGVycywgcHJhY3RpdGlvbmVycyBjYW4gb3B0aW1pemUKbW9kZWxzIHRvIGFjaGlldmUgaGlnaGVyIGFjY3VyYWN5LCBiZXR0ZXIgZ2VuZXJhbGl6ZSB0byB1bnNlZW4gZGF0YSwgYW5kCnByZXZlbnQgaXNzdWVzIGxpa2Ugb3ZlcmZpdHRpbmcgb3IgdW5kZXJmaXR0aW5nLiBUaGlzIHByb2Nlc3MgYWxsb3dzIGZvcgp0aGUgY3VzdG9taXphdGlvbiBvZiBtb2RlbHMgdG8gc3BlY2lmaWMgZGF0YSBzZXRzIGFuZCBvYmplY3RpdmVzLAplbmFibGluZyB0aGUgZGlzY292ZXJ5IG9mIHRoZSBiZXN0IGNvbmZpZ3VyYXRpb24gZm9yIGEgZ2l2ZW4gcHJvYmxlbS4KCiMjIENob29zaW5nIEh5cGVycGFyYW1ldGVyczogVmFsaWRhdGlvbiBTZXRzCgpUaGlzIGlzIHRoZSBiYXNpcyBmb3IgYSAqKnZhbGlkYXRpb24gc2V0KiouIElmIHdlIGhhZCBleHRyYSBoZWxkLW91dApkYXRhIHNldCwgd2UgY291bGQgdHJ5IGEgYnVuY2ggb2YgaHlwZXJwYXJhbWV0ZXJzIG9uIHRoZSB0cmFpbmluZyBzZXQsCmFuZCBzZWUgd2hpY2ggb25lIHJlc3VsdHMgaW4gYSBtb2RlbCB0aGF0IHBlcmZvcm1zIHRoZSBiZXN0IG9uIHRoZQp2YWxpZGF0aW9uIHNldC4gV2UgdGhlbiB3b3VsZCBjaG9vc2UgdGhhdCBoeXBlcnBhcmFtZXRlciwgYW5kIHVzZSBpdCB0bwpyZWZpdCB0aGUgbW9kZWwgb24gYm90aCB0aGUgdHJhaW5pbmcgZGF0YSBhbmQgdmFsaWRhdGlvbiBkYXRhLiBXZSBjb3VsZAp0aGVuLCBmaW5hbGx5LCBldmFsdWF0ZSBvbiB0aGUgdGVzdCBzZXQuCgohW1ZhbGlkYXRpb24gc2V0XSguLi9pbWFnZXMvdmFsaWRhdGlvbi5wbmcpCgojIENyb3NzIFZhbGlkYXRpb24KCldlIGp1c3QgZm9ybXVsYXRlZCB0aGUgcHJvY2VzcyBvZiBjaG9vc2luZyBhIGh5cGVycGFyYW1ldGVyIHdpdGggYQpzaW5nbGUgdmFsaWRhdGlvbiBzZXQuIEhvd2V2ZXIsIHRoZXJlIGFyZSBtYW55IHdheXMgdG8gcGVyZm9ybQp2YWxpZGF0aW9uLiBUaGUgbW9zdCBjb21tb24gd2F5IGlzICoqY3Jvc3MtdmFsaWRhdGlvbioqLgpDcm9zcy12YWxpZGF0aW9uIGlzIG1vdGl2YXRlZCBieSB0aGUgY29uY2VybiB0aGF0IHdlIG1heSBub3QgY2hvb3NlIHRoZQpiZXN0IGh5cGVycGFyYW1ldGVyIGlmIHdlJ3JlIG9ubHkgdmFsaWRhdGluZyBvbiBhIHNtYWxsIGZyYWN0aW9uIG9mIHRoZQpkYXRhLiBJZiB0aGUgdmFsaWRhdGlvbiBzYW1wbGUsIGp1c3QgYnkgY2hhbmNlLCBjb250YWlucyBzcGVjaWZpYyBkYXRhCnNhbXBsZXMsIHdlIG1heSBiaWFzIG91ciBtb2RlbCBpbiBmYXZvciBvZiB0aG9zZSBzYW1wbGVzLCBhbmQgbGltaXQgaXRzCmdlbmVyYWxpemFiaWxpdHkuCgpTbywgZHVyaW5nIGNyb3NzLXZhbGlkYXRpb24sIHdlIGVmZmVjdGl2ZWx5IHZhbGlkYXRlIG9uIHRoZSAqZW50aXJlKgp0cmFpbmluZyBzZXQsIGJ5IGJyZWFraW5nIGl0IHVwIGludG8gZm9sZHMuIEhlcmUncyB0aGUgcHJvY2VzczogV2UgY2FuCnVzZSBhIHByb2Nlc3MgY2FsbGVkIGNyb3NzLXZhbGlkYXRpb24gdG8gZG8gc2VsZWN0IHRoZSBiZXN0CgoxLiAgUGVyZm9ybSBhIHRyYWluLXRlc3Qgc3BsaXQsIGFzIHlvdSBub3JtYWxseSB3b3VsZC4KMi4gIENob29zZSBhIG51bWJlciBvZiBmb2xkcyAtIHRoZSBtb3N0IGNvbW1vbiBpcyAkSz01JCAtIGFuZCBzcGxpdCB1cAogICAgeW91ciB0cmFpbmluZyBkYXRhIGludG8gdGhvc2UgZXF1YWxseSBzaXplZCAiZm9sZHMiLgozLiAgRm9yIGVhY2ggaHlwZXJwYXJhbWV0ZXIgeW91IHdhbnQgdG8gdHVuZSwgc3BlY2lmeSB0aGUgcG9zc2libGUKICAgIHZhbHVlcyBpdCBjb3VsZCB0YWtlLiBUaGVuLCBmb3IgZWFjaCBoeXBlcnBhcmFtZXRlcjoKICAgIDEuICBGb3IgKmVhY2gqICp2YWx1ZSogb2YgdGhhdCBoeXBlcnBhcmFtZXRlciwgd2UncmUgZ29pbmcgdG8gZml0CiAgICAgICAgJEskIG1vZGVscy4gTGV0J3MgYXNzdW1lICRLPTUkLiBUaGUgZmlyc3QgbW9kZWwgd2lsbCBiZSBmaXQgb24KICAgICAgICBGb2xkcyAyLTUsIGFuZCB2YWxpZGF0ZWQgb24gRm9sZCAxLiBUaGUgc2Vjb25kIG1vZGVsIHdpbGwgYmUgZml0CiAgICAgICAgb24gRm9sZHMgMSwgMy01LCBhbmQgdmFsaWRhdGVkIG9uIEZvbGQgMi4gVGhpcyBwcm9jZXNzIGNvbnRpbnVlcwogICAgICAgIGZvciBhbGwgNSBzcGxpdHMuCiAgICAyLiAgVGhlIHBlcmZvcm1hbmNlIG9mIGVhY2ggdmFsdWUgb2YgdGhhdCBoeXBlcnBhcmFtZXRlciBpcwogICAgICAgIHN1bW1hcml6ZWQgYnkgdGhlIGF2ZXJhZ2UgcHJlZGljdGl2ZSBwZXJmb3JtYW5jZSBvbiBhbGwgNQogICAgICAgIGhlbGQtb3V0IGZvbGRzLiBXZSB0aGVuIGNob29zZSB0aGUgaHlwZXJwYXJhbWV0ZXIgdmFsdWUgdGhhdCBoYWQKICAgICAgICB0aGUgYmVzdCBhdmVyYWdlIHBlcmZvcm1hbmNlLgo0LiAgV2UgY2FuIHRoZW4gcmVmaXQgYSBuZXcgbW9kZWwgdG8gdGhlIGVudGlyZSB0cmFpbmluZyBzZXQsIHVzaW5nIG91cgogICAgY2hvc2VuIGh5cGVycGFyYW1ldGVyKHMpLiBUaGF0J3Mgb3VyIGZpbmFsIG1vZGVsIC0gZXZhbHVhdGUgaXQgb24KICAgIHRoZSB0ZXN0IHNldCEKCiFbY3Jvc3MtdmFsaWRhdGlvbl0oaHR0cHM6Ly9zY2lraXQtbGVhcm4ub3JnL3N0YWJsZS9faW1hZ2VzL2dyaWRfc2VhcmNoX2Nyb3NzX3ZhbGlkYXRpb24ucG5nKQoKIyMgSHlwZXJwYXJhbWV0ZXIgVHVuaW5nIGFuZCBDcm9zcyBWYWxpZGF0aW9uIGluIFByYWN0aWNlCgpXZSBuZWVkIHRvIGRvIHR3byB0aGluZ3M6CgoxLiAgRGVjaWRlIHRvIHBlcmZvcm0gaHlwZXJwYXJhbWV0ZXIgdHVuaW5nIG9uIHRoZSBwZW5hbHR5IHZhbHVlLCBhbmQKMi4gIERvIHNvIHVzaW5nIGNyb3NzLXZhbGlkYXRpb24uCgpUaGUgYHRpZHltb2RlbHNgIHN1aXRlIGhhcyB0d28gcGFja2FnZXMgdG8gaGVscCB1cyB3aXRoIHRoZXNlIHN0ZXBzOgpgdHVuZWAgYW5kIGByc2FtcGxlYC4KCkxldCdzIGlsbHVzdHJhdGUgYm90aCB0aGVzZSBwYWNrYWdlcyBpbiB0aGUgY2xhc3NpZmljYXRpb24gZXhhbXBsZS4gV2UKYWxyZWFkeSBoYXZlIGEgcmVjaXBlIHNldCB1cDoKCmBgYHtyfQp2b3RlcmVjaXBlCmBgYAoKV2hlbiBzcGVjaWZ5aW5nIHRoZSBtb2RlbCwgaG93ZXZlciwgd2UncmUgZ29pbmcgdG8gZG8gc29tZXRoaW5nIHNsaWdodGx5CmRpZmZlcmVudDoKCmBgYHtyfQp0dW5lZF9sb2dpc3RpY19tb2RlbCA8LSBsb2dpc3RpY19yZWcoCiAgcGVuYWx0eSA9IHR1bmUoKSwKICBlbmdpbmUgPSAiZ2xtbmV0IikKYGBgCgpXZSBwYXNzZWQgaW4gYSBmdW5jdGlvbiBjYWxsZWQgYHR1bmUoKWAuIFRoaXMgc2lnbmFscyB0byBgdGlkeW1vZGVsc2AKdGhhdCB3ZSdkIGxpa2UgdG8gdHVuZSB0aGlzIGh5cGVycGFyYW1ldGVyLiBIb3cgZG8gd2UgaW5kaWNhdGUgd2hhdAp2YWx1ZXMgd2Ugc2hvdWxkIHRlc3QgZHVyaW5nIHR1bmluZz8gV2UncmUgZ29pbmcgdG8ga2VlcAp0aGluZ3Mgc2ltcGxlIGFuZCBmb2N1cyBvbiB0aGUgbW9zdCBiYXNpYyBjaG9pY2Ugb2YgdHVuaW5nOiBhIGdyaWQKc2VhcmNoLiBJbiB0aGlzIGNhc2UsIHdlIHNwZWNpZnkgYSByYW5nZSBvZiB2YWx1ZXMsIGFuZCB0aGVuIHRlc3QgZXZlcnkKc2luZ2xlIG9uZSBmb3IgdGhlIGh5cGVycGFyYW1ldGVyLiBXZSB1c2UgdGhlIGBncmlkX3JlZ3VsYXJgIGZ1bmN0aW9uCmZvciB0aGlzIHByb2NlZHVyZToKCmBgYHtyfQojIENyZWF0ZSBncmlkIG9mIHBhcmFtZXRlcnMKY3ZfZ3JpZCA8LSBncmlkX3JlZ3VsYXIoCiAgcGVuYWx0eShyYW5nZSA9IGMoLTUsIDUpKSwgIyBzZWxlY3QgcGVuYWx0eSB2YWx1ZXMgd2l0aCAtNSBhbmQgNSAKICBsZXZlbHM9MTApICMgcGljayAxMCB2YWx1ZXMgYmV0d2VlbiAtNSBhbmQgNSAKCnByaW50KGN2X2dyaWQpCmBgYAoKIyMg8J+UlCBRdWVzdGlvbiA0OiBBbmFseXppbmcgYSBHcmlkCgpXZSBoYXZlIGEgaHlwZXJwYXJhbWV0ZXIgZ3JpZCB3aXRoIDEwIHJvd3MuIFdoYXQgZG9lcyBlYWNoIG9mIHRoZXNlCnNpZ25pZnk/CgoqKlNvbHV0aW9uIDQqKjogRWFjaCBpcyBhIHBvdGVudGlhbCBzZXQgb2YgaHlwZXJwYXJhbWV0ZXJzLCBvcgphcmd1bWVudHMsIGZvciBvdXIgbG9naXN0aWMgcmVncmVzc2lvbi4gUiB3aWxsIHRyeSBlYWNoIGNvbWJpbmF0aW9uIHRvCmZpbmQgdGhlIG9uZSB0aGF0IGdlbmVyYXRlcyB0aGUgbW9zdCBhY2N1cmF0ZSBwcmVkaWN0aW9uLiBBcyB3ZSBhZGQgbW9yZQpoeXBlcnBhcmFtZXRlcnMsIHRoZSBjb21iaW5hdGlvbnMgd2lsbCBpbmNyZWFzZSBleHBvbmVudGlhbGx5LgoKIyMg8J+OrCAqKkRlbW8qKjogUGVyZm9ybWluZyBIeXBlcnBhcmFtZXRlciBUdW5pbmcgd2l0aCBDcm9zcyBWYWxpZGF0aW9uCgpOZXh0LCB3ZSBuZWVkIHRvIHNwZWNpZnkgaG93IHdlIHdpbGwgcGVyZm9ybSBjcm9zcy12YWxpZGF0aW9uLiBGcm9tIHRoZQpgcnNhbXBsZWAgcGFja2FnZSwgd2UgY2FuIHVzZSB0aGUgZnVuY3Rpb24gYHZmb2xkX2N2YCB0byBjcmVhdGUgdGhlCnRyYWluaW5nIGZvbGRzLiBJbiB0aGlzIGNhc2UsIGB2YCBpcyB3aGF0J3MgdXNlZCBmb3IgIksiLgoKYGBge3J9CnZvdGVfZm9sZHMgPC0gdmZvbGRfY3Yodm90ZV90cmFpbiwgdiA9IDUpCnZvdGVfZm9sZHMKYGBgCgpXZSBoYXZlIGEgdHVuZWQgbW9kZWwsIGEgZ3JpZCBvZiBoeXBlcnBhcmFtZXRlcnMsIGFuZCBhIHNldCBvZiBmb2xkcy4gV2UKY3JlYXRlIG91ciB3b3JrZmxvdyBhcyBiZWZvcmU6IHdlIGlucHV0IHRoZSByZWNpcGUgKGkuZS4sIHRoZSBzZXQgb2YgcHJlLXByb2Nlc3NpbmcgaW5zdHJ1Y3Rpb25zKSBhbmQgdGhlIG1vZGVsIHNwZWNpZmljYXRpb24uIEJ1dCwgdG8gdHJhaW4gdGhlIHdvcmtmbG93LCB3ZSB1c2UgdGhlIGB0dW5lX2dyaWRgIGZ1bmN0aW9uLiBBbGwgdGhlIHBpZWNlcyB3ZSd2ZSBjcmVhdGVkIGFyZSBwYXNzZWQgaW50byB0aGlzCmZ1bmN0aW9uOgoKYGBge3J9CiMgQ3JlYXRlIHdvcmtmbG93CnZvdGVfd2Zsb3cgPC0gd29ya2Zsb3coKSAlPiUKICBhZGRfcmVjaXBlKHZvdGVyZWNpcGUpICU+JQogIGFkZF9tb2RlbCh0dW5lZF9sb2dpc3RpY19tb2RlbCkKCiMgVHVuZSBhbmQgZml0IG1vZGVscyB3aXRoIHR1bmVfZ3JpZCgpCnZvdGVfY3ZfZml0IDwtIHR1bmVfZ3JpZCgKICAjIFRoZSB3b3JrZmxvdwogIHZvdGVfd2Zsb3csIAogICMgVGhlIGZvbGRzIHdlIGNyZWF0ZWQKICByZXNhbXBsZXMgPSB2b3RlX2ZvbGRzLAogICMgVGhlIGdyaWQgb2YgaHlwZXJwYXJhbWV0ZXJzCiAgZ3JpZCA9IGN2X2dyaWQpCmBgYAoKVGhlcmUgYXJlIHNvbWUgbmljZSBwbG90dGluZyBmdW5jdGlvbnMgd2UgY2FuIHVzZSB0byB2aXN1YWxpemUgaG93IHRoZQpwZXJmb3JtYW5jZSB2YXJpZXMgYXMgYSBmdW5jdGlvbiBvZiB0aGUgcmVndWxhcml6YXRpb24uIEZvciBleGFtcGxlLApjaGVjayBvdXQgdGhlIGBhdXRvcGxvdCgpYCBmdW5jdGlvbjoKCmBgYHtyfQphdXRvcGxvdCh2b3RlX2N2X2ZpdCwgCiAgICAgICAgIG1ldHJpYz0iYWNjdXJhY3kiKSAKYGBgCgpFYWNoIG9mIHRoZSB0ZW4gcG9pbnRzIG9uIHRoaXMgZ3JhcGggY29ycmVzcG9uZHMgdG8gYSBzZXQgb2YKaHlwZXJwYXJhbWV0ZXJzLiBXZSBjYW4gc2VlIHRoYXQgdGhlIHRoaXJkIGNvbWJpbmF0aW9uIHlpZWxkcyB0aGUgaGlnaGVzdCBhY2N1cmFjeQpvZiBhYm91dCA4MS41JS4gV2hhdCBkb2VzIHRoaXMgdGVsbCB1cyBhYm91dCBob3cgbXVjaCByZWd1bGFyaXphdGlvbiB3ZQpzaG91bGQgdXNlPwoKSW5zdGVhZCBvZiBwaWNraW5nIHRoZSBjb21iaW5hdGlvbiBvZiBoeXBlcnBhcmFtdGVycyB0aGF0IHlpZWxkcyB0aGUKYmVzdCBhY2N1cmFjeSBieSBleWUsIHdlIGNhbiBhdXRvbWF0ZSB0aGlzIHByb2NlZHVyZS4gVGhlIGBzZWxlY3RfYmVzdGAKZnVuY3Rpb24gd2lsbCBkbyB0aGlzIGZvciB1czoKCmBgYHtyfQojIFNlbGVjdCBiZXN0IG1ldHJpYyBhY2NvcmRpbmcgdG8gYWNjdXJhY3kgCnZvdGVfY3ZfYmVzdCA8LSBzZWxlY3RfYmVzdCh2b3RlX2N2X2ZpdCwgbWV0cmljID0gImFjY3VyYWN5IikKdm90ZV9jdl9iZXN0CmBgYAoKQ3Jvc3MtdmFsaWRhdGlvbiBzZWxlY3RlZCBhIHBlbmFsdHkgdmFsdWUgb2YgMC4wMDE3IGFzIHRoZSBiZXN0IGh5cGVycGFyYW1ldGVyIGZvciBvdXIgbW9kZWwuIFdoYXQgZG8gd2UgZG8gYXQgdGhpcyBwb2ludD8KClJlY2FsbCB0aGF0LCBkdXJpbmcgY3Jvc3MtdmFsaWRhdGlvbiwgd2Ugc3BsaXQgdXAgdGhlIGRhdGEgYW5kIGV4YW1pbmUKcGVyZm9ybWFuY2UgYWNyb3NzIG1hbnkgZm9sZHMuIE5vdyB0aGF0IHdlIGtub3cgd2hhdCBpcyBsaWtlbHkgdGhlIGJlc3QKcGVuYWx0eSwgd2UgY2FuIHJlLXRyYWluIG9uIHRoZSAqZW50aXJlKiB0cmFpbmluZyBzZXQgdG8gcHJvZHVjZSBvdXIgZmluYWwgbW9kZWwuCgpXZSBkbyB0aGlzIHdpdGggdGhlIGBmaW5hbGl6ZV93b3JrZmxvd2AgZnVuY3Rpb24uCgpgYGB7cn0KIyBHZXQgb3VyIGZpbmFsIG1vZGVsIGFuZCBmaW5hbGl6ZSB3b3JrZmxvdwpjdl9maW5hbCA8LSB2b3RlX3dmbG93ICU+JQogIGZpbmFsaXplX3dvcmtmbG93KHBhcmFtZXRlcnMgPSB2b3RlX2N2X2Jlc3QpICU+JSAjIHVzZSB0aGUgYmVzdCBwYXJhbWV0ZXJzIAogIGZpdChkYXRhID0gdm90ZV90cmFpbikKY3ZfZmluYWwgJT4lIHRpZHkoKQpgYGAKCkFuZCBsYXN0bHksIHdlJ2xsIGV4YW1pbmUgdGhlIHBlcmZvcm1hbmNlIG9uIHRoZSB0ZXN0IHNldDoKCmBgYHtyfQpjdl9wcmVkaWN0aW9ucyA8LSBhdWdtZW50KGN2X2ZpbmFsLCBuZXdfZGF0YSA9IHZvdGVfdGVzdCkKCmFjY3VyYWN5KGN2X3ByZWRpY3Rpb25zLCB0cnV0aCA9IHZvdGVkLCBlc3RpbWF0ZSA9IC5wcmVkX2NsYXNzLCB0eXBlPSJjbGFzcyIpCmBgYAoKIyMg8J+UlCBRdWVzdGlvbiA1OiBBY2N1cmFjeSBvbiB0aGUgVGVzdCBTZXQKCk5vdGljZSBvdXIgZmluYWwgYWNjdXJhY3kgb24gdGhlIHRlc3QgaXMgYWJvdXQgODAuODUlLCB3aGVyZWFzIG91cgphY2N1cmFjeSBvbiB0aGUgdHJhaW5pbmcgc2V0IHdhcyBoaWdoZXIgYXQgODEuNSUuIFdoeSBpcyBvdXIgdGVzdAphY2N1cmFjeSBsb3dlcj8KCioqU29sdXRpb24gNSoqOiBXZSBzZWxlY3RlZCBvdXIgaHlwZXJwYXJhbWV0ZXJzIChpLmUuLCB0aGUgcGVuYWx0eQpmdW5jdGlvbikgdG8geWllbGQgdGhlIGhpZ2hlc3Qgb3V0cHV0ICpzcGVjaWZpY2FsbHkqIG9uIG91ciB0cmFpbmluZyBzZXQuIAoKIyMgSHlwZXJwYXJhbWV0ZXIgVHVuaW5nIHZzLiBEZWZhdWx0IEFyZ3VtZW50cwoKSWYgd2UgY29tcGFyZSB0aGlzIHRvIHRoZSBvcmlnaW5hbCBtb2RlbCB3ZSBmaXQgZWFybGllciB3aGVyZSB3ZSBtYWRlIGEKZ3Vlc3MgYXQgdGhlIGJlc3QgdmFsdWUgZm9yIHRoZSBwZW5hbHR5IGh5cGVycGFyYW1ldGVyLCBpdCByZXZlYWxzIHRoYXQKd2UgdGhhdCB3ZSBjYW4gZml0IGJldHRlciBtb2RlbHMgdmlhIGNyb3NzLXZhbGlkYXRpb24gdGhhbiB3aXRoIGp1c3QgYQpzaW5nbGUgdHJhaW5pbmcgc2V0LiBUaGlzIGlzIHJlZmxlY3RlZCBpbiB0aGUgaGlnaGVyIGFjY3VyYWN5IGNvbXBhcmVkCndpdGggdGhlIG1vZGVsIHdoZXJlIHdlIHVzZWQgdGhlIGRlZmF1bHQgYXJndW1lbnRzIG9mIGBsb2dpc3RpY19yZWcoKWAuCgojIENvbmNsdWRpbmcgUmVtYXJrcwoKQ29uZ3JhdHVsYXRpb25zLCB5b3UndmUgbWFkZSBpdCEgV2UgY292ZXJlZCB0aGUgYmFzaWNzIG9mIHN1cGVydmlzZWQKbWFjaGluZSBsZWFybmluZyBpbiBgdGlkeW1vZGVsc2AgaW4gdGhpcyB3b3Jrc2hvcC4gSG93ZXZlciwgdGhlcmUncyBtdWNoCm1vcmUgdG8gZXhwbG9yZS4gVGhlIGJlc3Qgd2F5IHRvIGtlZXAgcHVzaGluZyBmb3J3YXJkIGlzIHRvIGNob29zZSBhCnByb2JsZW0gdG8gc3R1ZHksIGFuZCByZWZlciB0byB0aGUgZG9jdW1lbnRhdGlvbiB3aGVuIHlvdSBuZWVkIGhlbHAuIFRoZQp3ZWJzaXRlIFtLYWdnbGVdKGh0dHBzOi8vd3d3LmthZ2dsZS5jb20vKSBoYXMgbWFueSBnb29kIGRhdGEgc2NpZW5jZSBwcm9ibGVtcyB0byB3b3JrIG9uCmlmIHlvdSBuZWVkIGhlbHAgY2hvb3NpbmcgYSB0YXNrIQo=